Brain tumour segmentation with incomplete data

Brain tumour segmentation remains a challenging task, complicated by the marked heterogeneity of imaging appearances and their distribution across multiple modalities: FLAIR, T1-weighted, T2-weighted, and contrast-enhanced T1-weighted sequences (T1CE). This has compelled a research focus on uniformly multimodal models trained on complete acquisition sets rare in real-world clinical practice. Consider, for example, patients with renal failure who cannot receive contrast, artefact-spoiled sequences, or patients undergoing single-sequence intraoperative imaging. How well do segmentation models perform with such incomplete data, and what features of the lesion are identifiable under these circumstances? In a large-scale analysis involving 30 distinct segmentation models, we answer these questions with a state-of-the-art tumour segmentation modelling ensemble, nnU-Net-derived (Isensee et al, Nature Methods, 2020), deployed across all possible combinations of imaging modalities, trained, and tested with five-fold cross-validation on the 2021 BraTS-RSNA glioma population of 1251 patients. Segmentation performances for whole lesions range from Dice scores of 0.907 (single sequence) to 0.945 (full datasets) (Figure 1). When segmenting lesions by tissue type (enhancing tumour, non-enhancing tumour and oedema), Dice scores range from 0.701 (single sequence) to 0.891 (full datasets). Models missing postcontrast imaging still achieve a Dice coefficient for the whole tumour of 0.942 and identify the enhancing tumour component with Dice of up to 0.790 (Figure 2). Segmentation models can identify tumours with missing data, and can be used in clinical situations where partial data is frequent

The legibility of the imaged human brain

Our knowledge of the organisation of the human brain at the population-level is yet to translate into power to predict functional differences at the individual-level, limiting clinical applications, and casting doubt on the generalisability of inferred mechanisms. It remains unknown whether the difficulty arises from the absence of individuating biological patterns within the brain, or from limited power to access them with the models and compute at our disposal. Here we comprehensively investigate the resolvability of such patterns with data and compute at unprecedented scale. Across 23810 unique participants from UK Biobank, we systematically evaluate the predictability of 25 individual biological characteristics, from all available combinations of structural and functional neuroimaging data. Over 4526 GPU*hours of computation, we train, optimize, and evaluate out-of-sample 700 individual predictive models, including multilayer perceptrons of demographic, psychological, serological, chronic morbidity, and functional connectivity characteristics, and both uni- and multi-modal 3D convolutional neural network models of macro- and micro-structural brain imaging. We find a marked discrepancy between the high predictability of sex (balanced accuracy 99.7%), age (mean absolute error 2.048 years, R2 0.859), and weight (mean absolute error 2.609Kg, R2 0.625), for which we set new state-of-the-art performance, and the surprisingly low predictability of other characteristics. Neither structural nor functional imaging predicted individual psychology better than the coincidence of common chronic morbidity (p<0.05). Serology predicted common morbidity (p<0.05) and was best predicted by it (p<0.001), followed by structural neuroimaging (p<0.05). Our findings suggest either more informative imaging or more powerful models will be needed to decipher individual level characteristics from the brain.Comment: 36 pages, 6 figures, 1 table, 2 supplementary figure